The majority of measurement experiments in modern science use the theory of experiment design. While an experimental method supplies a methodology for establishing a body of reliable facts and analyses which can aid in answering certain clearly stated questions relevant to an experimenter, an experiment design is the plan for collecting and analyzing data to answer certain clearly stated questions relevant to the needs of the experimenter. The experiment design is based on contrasting two or more treatment conditions, where ideally the subjects are treated in identical manners, except for one feature that is different. An observed difference in response can then be attributed to the designed differences among the treatment conditions. A well designed experiment permits the inference of facts from a small number of observations that might otherwise require an exhaustive set of observations. The results of an experiment can indicate how to change an average response of a system to a different desired value in response to a treatment. Alternately, the results of an experiment can show how to reduce variations in a process, help to make a process more robust or even help to distinguish which variables are important to control. Experimental investigations are generally iterative in nature, and the experimenter generally uses the results of, e.g., a screening process, to decide on the factors that are important to control in the following experimental design. Some basics of the design of experiments are summarized at trochim.human.cornell.edu/kb/desintro.htm. There are also commercially available texts that provide a discussion of the design of experiments, such as Barrentine, An Introduction to Design of Experiments: A Simplified Approach, ASQ Quality Press, 1999.
An experiment design provides the structuring of an experiment for the application of a number of treatments to one or more experimental units, i.e., a number of different subjects on which the experiment is performed (e.g., animals or cells). The factors, which form the independent variables, are the controllable settings in the experiment, and are typically represented by Xs, i.e. X1, X2, X3, etc. Each factor can have one or more values, subdivisions or settings, each referred to as a level of the factor, an example being the different concentrations of a drug. The experiment design specifies the levels for the factors for each performance of the experiment. A response value or setting forms the dependent variable, and is the measured outcome from the performance of an experiment. In an experiment design, the factors are chosen by the experimenter in the belief that they should affect the response value being monitored. To increase the confidence level in the results of an experiment, the experimenter can choose to form a number of replications.
A replication is a repetition of some or all treatments on two or more experiment units. If all of the experiments are repeated three times, the study is said to have three replications. Replications of experiments are necessary to provide an indication of the amount of the experiment error in the measurement of any response value. An experiment design specifies the number of experiments, the factor-level combination for each of the experiments, and the number of replications of the experimental study. The different response values or settings from the different replications of the scans performed, also called replicates, are used to calculate a statistical average of response to the application of a factor. For balance, it is preferable to have an equal number of observations for each treatment. This helps to minimize the possibility of some types of bias. Increasing the number of replicates can help increase the statistical confidence in the correlations of the responses between levels, such as a confidence level of 95%. An example of a software tool that can compute a minimal number of replicates is BioMine™ 1.0 Experiment Design Tool™ (available from Gene Network Sciences, Inc., Ithaca, N.Y.).
An experiment design can also indicate if there is any interaction between factors when the levels of the factors are applied in combination in an experimental design. Two factors A and B are said to interact if the effect of one depends on the level of the other, such as if the factors are time and dosage levels of a drug. An interaction graph can help to visualize interactions between two factors, e.g., if the response variable is plotted as the y-axis, the x-axis is the levels of one of the factors, and each curve plotted on the interaction graph represents the responses to the levels of the other factor. A main effect is an outcome where there is a consistent difference between levels of a factor on an interaction graph. For example, there is a main effect from a factor such as a drug if there is a statistical difference between the average response for the different drug dosage levels at all levels of time in an experimental study.
FIGS. 1A, 1B and 1C illustrate interaction graphs for an experiment design where the two factors of drugs and time are applied, each at two different levels. The levels of the factor time on the x-axis and 1 hour and 4 hours, while the levels of the factor drugs on the x-axis are drug dosages of level 1 ng/ml and 10 mg/ml. FIG. 1A depicts an interaction graph where there is no effect of any treatment condition, since the effect of the drug and time are the same. FIGS. 1B and 1C depict interaction graphs where there is a main effect only for time, or only for drugs, respectively. FIG. 1B shows that, for all drug dosage levels, the 4 hours condition has a greater effect than the 1 hour condition. FIG. 1C shows that the 10 ng/ml dosage level yields a greater effect than the 1 ng/ml level, for any given amount of time. The interaction graphs can also show a main effect for both factors. FIG. 2A illustrates an interaction graph for the case where there is a main effect for both factors, but with no interaction. FIGS. 1A, 1B, 1C and 2A all show that, if there is no interaction, the curves on the graph are basically parallel. Any amount of interaction will be exhibited by some amount of deviation from a parallel arrangement. FIG. 2B, illustrates the case where there is an interaction between the factors drug and time, but only for the case where time is 4 hours and drug is 10 ng/ml. FIG. 2B illustrates a case of a more complex “cross-over” interaction between the two factors, where the combinations of 1 hour and 1 ng/ml or 4 hours and 10 ng/ml worked equally as well, while the other combinations do poorly. A poorly designed experiment could also lead to confounding, which is an inability to attribute a change in a response variable to a factor. A good experiment design works to reduce the incidence of confounding.
There are many different statistical analysis methods that can be used to analyze the measurements derived from any particular experiment design chosen. In many statistical tests, any measurement error is required to be independent of the measurement quantity (constant measurement variance). Analysis of Variance (ANOVA) is an example (see, e.g., Statistics For Experimenters, Box, Hunter and Hunter, John Wiley and Sons, 1978; Siegel et al., Nonparametric statistics for the behavioural sciences, McGraw Hill, 2nd edition, 1998; Conover, Practical Nonparametric Statistics, John Wiley and Sons, 3rd edition, 1998; Altman, Practical Statistics for Medical Research, CRC Press, 1991; Berry, Statistical Methods in Medical Research, Blackwell Science, Inc., 2001). ANOVA is a method for detecting whether there are statistical differences among the mean of different measurement groups. ANOVA can be used to determine whether there is a statistically significant difference in protein expression data between or within groups of genes. Examples of standard statistical techniques applied to analyze the measured results of an experiment design include t-tests (paired and unpaired), one-way or two-way ANOVA, factorial and fractional factorial designs (e.g., two-level designs), the method of least squares (linear or nonlinear models), and response surface methodology. The statistical analysis methods are used to interpret the data derived from the experiment design, i.e., to indicate if any observed difference in the main effects between groups from the one or more factors is actually statistically valid.
There are software tools available in the art for storing and manipulating data derived from various experiments in biotechnology, or for performing some statistical analysis of the data. BioMine™ 1.0 (see gnsbiotech.com/biomine.shtml) is an application for the analysis of gene expression data that provides for data importation and normalization (from e.g., replicates). The application also provides for some manipulation of the normalized data, e.g., an Experiment Design Tool™ that computes the minimal number of replicates for statistically validating an experiment result within a given confidence level. Rosetta Biosoftware provides a flexible, manipulable searcheable database, i.e., the Rosetta Biosoftware Resolver® gene expression data analysis system (Rosetta BioSoftware, Kirkland, Wash.), which can be used to store, and search a compilation of, e.g., gene expression data sets. Silicon Genetics (Redwood City, Calif.) provides different software tools for raw data normalization, analysis, and visualization, called GeneSpring™ software version 5.0, which is a visualization software package for, e.g., microarray data, and GeNet™ software version 3.0, which is provides for data archiving and retrieval by assigning attributes to gene expression profiles. Affymetrix, Inc. (Santa Clara, Calif.) provides the Affymetrix® Analysis Data Model (AADM), a relational database schema used to store Affymetrix® GeneChip expression results, and Data Mining Tool (DMT) software tools for filtering and sorting GeneChip® array data stored in an Affymetrix® Analysis Data Model (AADM)-compatible database generated by using the Laboratory Information Management System (LIMS) or MicroDB™ (which creates Affymetrix® Analysis Data Model (AADM)-compliant databases from experimental data sets derived from GeneChip® or spotted arrays).
While the Rosetta Resolver® software tools provided a searchable database for gene expression data analysis, the software package does not provide a user interface (UI) tool for processing experiment data according to an experiment definition, or the capability for saving the experiment definition instructions on how to process the data. Prior implementations of analysis tools in currently available systems, e.g., Rosetta Resolver®, require the user to go through many labor-intensive steps, such as multiple data searches, managers, and complex wizards to arrive at a result. As an example, a user has to search for different components in different parts of the application. For instance, profiles related to a project are searched among all profiles, experiments among all experiments, cluster results among all clusters, etc. Additionally, currently available systems treat each profile separately, and thus it is difficult to assign different profiles to more than one treatment group.
Given the above background, there remains a need for an experiment definition system that can digitally reflect an experiment design. Such a system should provide the experimenter with a user-interface that allows digital manipulation, organization and analysis of the results of any number of different measurements according to any desired experiment design, and provide processing and analysis pipelines for implementing the experiment definitions. The experiment definition system should also be able to store the experiment definitions customized by the user for further use. Such an experiment definition system would offer increased flexibility over the existing software tools.